An official website of the United States government
Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher.
Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?
Some links on this page may take you to non-federal websites. Their policies may differ from this site.
-
The deep linear network (DLN) is a model for implicit regularization in gradient based optimization of overparametrized learning architectures. Training the DLN corresponds to a Riemannian gradient flow, where the Riemannian metric is defined by the architecture of the network and the loss function is defined by the learning task. We extend this geometric framework, obtaining explicit expressions for the volume form, including the case when the network has infinite depth. We investigate the link between the Riemannian geometry and the training asymptotics for matrix completion with rigorous analysis and numerics. We propose that under small initialization, implicit regularization is a result of bias towards high state space volume.more » « less
-
We construct Dyson Brownian motion for β ∈ (0, ∞] by adapting the extrinsic construc- tion of Brownian motion on Riemannian manifolds to the geometry of group orbits within the space of Hermitian matrices. When β is infinite, the eigenvalues evolve by Coulombic repulsion and the group orbits evolve by motion by (minus one half times) mean curvature.more » « less
